Study of B meson decays to three-body charmless hadronic final states 
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Wo report results of a study of charmless B meson decays to three-body Ktvtt, KKtv and KKK fi- 
nal states. Measurements of branching fractions for B decays to K'^^^\'^'k~ , K'^ K~ , K'^ K~ , 
K+K%K% and K%K%K% final states are presented. The decays B° K^K+K', B+ K+K°sK°s 
and B^ K'^^K%K'^ are observed for the first time. We also report evidence for B'^ — » K'^ K~'k'^ 
decay. For the three-body final states K^K+i^-, KIkIt^^ , K+K+n' and K-n+Tr+ 90% confi- 
dence level upper limits are reported. Finally, we discuss the possibility of using the three-body 
B° — > K%K~^K~ decay for CP violation studies. The results are obtained with a 78fb~^ data 
sample collected at the T(4S') resonance by the Belle detector operating at the KEKB asymmetric 
energy e+e^ collider. 

PACS numbers; 13.25.Hw, 14.40.Nd 



INTRODUCTION 

Studies of three-body charmless hadronic final states are a natural extension of studies of two-body final states. 
Some of the final states considered so far as two-body (for example fm, K*n, etc.) are, in fact, quasi-two-body since 

they produce a wide resonance state that immediately decays, in the simplest case, into two particles producing a 
three-body final state. Multiple resonances occurring nearby in phase space will interfere and a full amplitude analysis 
is required to extract correct branching fractions for the intermediate quasi-two-body states. B meson decays to three- 
body charmless hadronic final states may also provide new possibilities for CP violation searches. For example, a 
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new method to extract the weak angle (pj, from isospin analysis and measurement of time dependent asymmetry in 
the decay B'^ —>■ i^^Tr+Tr" has been recently suggested in Ref. 0. Charmless decays of B mesons are also important 
in current searches for physics beyond the Standard Model (SM). Among three-body charmless final states, the 
5+ K^K^TT^ and B^ K^tt^tt^ decays, which proceed via b ssd and b — > dds transitions, respectively, 
provide a good opportunity to search for new physics. The SM prediction for the branching fraction of the decay 
£?+ K^K^n^ is of order 10^^^ and even smaller for the i4r~7r+7r+ final state However, there are extensions of 
the SM where these branching fractions can be enhanced up to 10""^ lEI^- Upper limits on branching fractions for 
these final states can be used to constrain parameters in some extensions of the SM Q . 

B meson decays to i^+^^V+vr^ and K^K^K^ final states have already been observed by the Belle, BaBar and 
CLEO experiments 0, 0, ■ -f'o^ other three-body charmless hadronic final states considered in this paper only 
upper limits have been reported so far In this paper, we present updated results on a study of B meson decays 
to three-body Kmr, KKtt and KKK final states. We also describe an isospin analysis of the decays of B mesons to 
three-kaon final states and discuss the use of the B^ KgK~^K~ three-body decay for CP violation measurements. 

The analysis is based on a 78fb~^ data sample, which contains 85.0 million BB pairs, collected with the Belle 
detector operating at the KEKB asymmetric-energy e+e^ collider Q with a ccnter-of-mass (cm.) energy at the 
T(45') resonance. The beam energies are 3.5 GeV for positrons and 8.0 GeV for electrons. For the study of the 
e+e" qq continuum background we use 8.3 fb~^ of data taken 60 MeV below the T(4S') resonance. The results 
presented here include the previous data and supersede the results on three-body charmless hadronic final states 
reported in Ref. 



THE BELLE DETECTOR 

The Belle detector is a large-solid-angle magnetic spectrometer based on a 1.5 T superconducting solenoid 
magnet. Charged particle tracking is provided by a three-layer silicon vertex detector and a 50-layer central drift 
chamber (CDC) that surround the interaction point. The charged particle acceptance covers laboratory polar angle 
between 6 — 17° and 150°, corresponding to about 92% of the full solid angle in the cm. frame. The momentum 
resolution is determined from cosmic rays and e+e^ — > /i^/i^ events to be ap^/pt ~ (0.30 0.19pt)%, where pt is the 
transverse momentum in GeV/c. 

Charged hadron identification is provided by dE/dx measurements in the CDC, an array of 1188 aerogel Cerenkov 
counters (ACC), and a barrel-like array of 128 time-of-flight scintillation counters (TOF); information from the 
three subdetectors is combined to form a single likelihood ratio, which is then used in kaon and pion selection. At 
large momenta (> 2.5 GeV/c) only the ACC and CDC are used to separate charged pious and kaons since here the 
TOF provides no additional discrimination. Electromagnetic showering particles are detected in an array of 8736 
CsI(Tl) crystals that covers the same solid angle as the charged particle tracking system. The energy resolution for 
electromagnetic showers is ge/E — (1.3 0.07/i? 0.8/£'^/^)%, where E is in GeV. Electron identification in Belle 
is based on a combination of dE/dx measurements in the CDC, the response of the ACC, and the position, shape 
and total energy deposition (i.e., E/p) of the shower detected in the calorimeter. The electron identification efficiency 
is greater than 92% for tracks with piab > 1-0 GeV/c and the hadron misidentification probability is below 0.3%. 
The magnetic field is returned via an iron yoke that is instrumented to detect muons and if^ mesons. We use a 
GEANT-based Monte Carlo (MC) simulation to model the response of the detector and determine acceptance [ll|. 



EVENT SELECTION 

Charged tracks are selected with a set of track quality requirements based on the average hit residual and on the 
distances of closest approach to the interaction point (IP). We also require that the track momenta transverse to 
the beam be greater than 0.1 GeV/c to reduce the low momentum combinatorial background. For charged kaon 
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identification, we impose a requirement on the particle identification variable, which has 86% efficiency and a 7% fake 
rate from misidentified pions. Charged tracks that are positively identified as electrons or protons are excluded. Since 
the muon identification efficiency and fake rate vary significantly with the track momentum, we do not veto muons 
to avoid additional systematic errors. 

Neutral kaons are reconstructed via the decay K^{K'^) tt+tt^. The invariant mass of the two pions is required 
to be within 12 MeV/c^ of the nominal mass. The displacement of the tt+tt" vertex from the IP in the r-cj) plane 
is required to be greater than 0.1 cm and less than 20 cm. The direction of the combined pion pair momentum in the 
r-(f> plane must be within 0.2 rad of the direction from the IP to the displaced vertex. 

We reconstruct B mesons in three-body Ktttt, KKtt and KKK final states, where K stands for a charged or neutral 
kaon, and tt for a charged pion. The inclusion of the charge conjugate mode is implied throughout this report. The 
candidate events are identified by their cm. energy difference, /S.E = (J^i^i) ^ ^boami and the beam constrained 
mass, Mbc — V-E-bcam ~ (l2iPi)'^' where -Ebcam = •\/s/2 is the beam energy in the cm. frame, and pi and Ei are the 
cm. three-momenta and energies of the candidate B meson decay products. We select events with Mbc > 5.20 GeV/c^ 
and —0.30 < AE < 0.50 GeV. For subsequent analysis, we also define a Mbc signal region of |Afbc — Mb| < 9 MeV/c^. 

To determine the signal yield, we use events with Afbc in the signal region and fit the AE distribution with the sum 
of a signal and a background function. The AE signal is parameterized by the sum of two Gaussian functions with 
the same mean. The widths and the relative fractions of the two Gaussians are determined from the MC simulation. 
We find that the signal MC events have a 10% narrower AE width than seen in the data. To correct for this effect, 
we introduce a scale factor that is determined from the comparison of the AE widths for _B+ — s- D'^tt^ events in 
MC and experimental data. The AE shape of the BB produced background is determined from MC simulation, as 
described below. The background from e+e^ ^ QQ (q — d, s and c quarks) continuum events is represented by a 
linear function. 



BACKGROUND SUPPRESSION 

An important issue for this analysis is the suppression of the large combinatorial background, which is dominated 
by qq continuum events. We suppress this background using variables that characterize the event topology. A more 
detailed description of the background suppression technique can be found in Ref . . 

Since the two B mesons produced from an T(4S') decay are nearly at rest in the cm. frame, their decay products 
are uncorrelated and events tend to be spherical. In contrast, hadrons from continuum qq events tend to exhibit a 
two-jet structure. We use 6'thr, which is the angle between the thrust axis of the B candidate and that of the rest of 
the event, to discriminate between the two cases. The distribution of | cos^thrl, shown in Fig.^a), is strongly peaked 
near jcos^thrl = 1.0 for qq events and is nearly flat for BB events. We require | cos 6'thr | < 0.80 for all three-body 
final states; this eliminates about 83% of the continuum background and retains 79% of the signal events. 

After imposing the cos^thr requirement, the remaining qq and BB events still have some differences in topology 
that are exploited for further continuum suppression. A Fisher discriminant |l2l | is formed from 11 variables: nine 
variables that characterize the angular distribution of the momentum flow in the event with respect to the B candidate 
thrust axis the angle of the B candidate thrust axis with respect to the beam axis, and the angle between the B 
candidate momentum and the beam axis. The discriminant, J- , is the linear combination of the input variables that 
maximizes the separation between signal and background. The coefficients are determined using below-resonance data 
and a large set of _B+ — > K^tt^tt^ signal MC events. We use the same set of coefficients for all three-body final states. 
Figure n^b) compares the T distributions for the _B+ K^tt^tt^ signal MC events, B^ D^tt^ events in data 
and and qq background events in below-resonance data. The separation between the mean values of the discriminant 
for the signal and background is approximately 1.3 times the signal width. For all Kmr and KKtt final states we 
impose a requirement !F > 0.8 on the Fisher discriminant variable that rejects about 90% of the remaining continuum 
background and retains 54% of the signal. The continuum background in the threc-kaon final states is much smaller 
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FIG. 1; Distribution of (a) |cosSthr| and (b) Fisher discriminant for the — > K^-k^-r signal MC events (histogram), 
-B^ — > D'^tv'^ events in data (open circles) and qq background in below-resonance data (filled circles). 



and a looser requirement > is imposed to retain the efficiency. This requirement rejects 53% of the remaining qq 
background and retains 89% of the signal. 

We also consider backgrounds that come from other B decays. We subdivide this background into two types. The 
first type is the background from decays that are dominantly b c tree transition. The description of these decays 
is taken from an updated version of the CLEO group event generator . The other potential source of background 
is charmless B decays that proceed via b — s- s(d) penguins or b u tree transitions. We studied a large set of 
potentially serious backgrounds from two-, three-, and four-body final states. Since the background from other B 
decays is substantially mode dependent, we give a detailed description of this type of background for each final state 
in the following sections. 

We find that the dominant background to the X+^'^V+Tr^ final states from other B decays is due to the B —> Dn, 
D ^ Kn decays and due to B+ J/ip{'ilj{2S))K+ decays, where J/'tp{ip{2S)) /i+M"- To exclude the B Dt: 
events, we apply the requirement on the Ktt invariant mass \M{KTr) — Md\ > 0.10 GeV/c^, where Md is the 
world average value for mass of the Z?°^+) meson 0. To suppress the background due to tt/K misidentification in 
the /f+TT+TT^ final state, we also exclude candidates if the invariant mass of any pair of oppositely charged tracks 
from the B candidate is consistent with the K^tt^ hypothesis within 15 MeV/c^ (^ 2.5cr), independently 

of the particle identification information. Modes with a J/'0(^(2S')) in the final state contribute due to muon-pion 
misidentification; the contribution from the J/ip{il;{2S)) e~^e~ submode is found to be negligible (less than 0.5%) 
after the electron veto requirement. We exclude J/ip{^p{2S)) signals by requirii ig \ M(h~^h~) — Mj/^\ > 0.07 GeV/c^ 
and |M(/i+/i^) — Af^(25)| > 0.05 GeV/c^, where /i+ and are pion candidates [l5j. Finally, we also reject candidates 
if the TT+TT^ invariant mass is within 50 MeV/c^ of the world average Xco mass The most significant background 
from charmless B decays is found to orig inate from the r;'A"+(°) followed by rj' TT+TT 7 decays. Another 

contribution comes from the B~^ — > p'^n'^ final state, where one of the final state pions is misidentified as a kaon. 
There is also a background from two-body charmless B Ktt decays. Although this background is shifted by about 
0.2 GeV from the AE signal region, it is important to take it into account to estimate correctly the background from 
qq continuum events. 
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FIG. 2: AE distributions for (a) B+ K+tt+tt- and (b) B° K° -K^TT . Points with error bars represent data, the open 
histogram is the result of the fit, and the hatched histogram shows the total background level. The straight line indicates the 
qq continuum background contribution. 



The AE distributions for selected K+^°\+tt- candidates that pass all the selection requirements are 

shown in Fig.|21 A significant enhancement in the B signal region is observed for both final states. Results of the fits 
are shown as open histograms in Fig. [21 and hatched histograms represent the background. While fitting the data we 
fix the shape of the BB background and allow its normalization to float. For background from charmless B decays 
we fix the normalization as well. Both the normalization and the slope of the qq background are floated during the 
fit. There is a large increase in the level of the BB related background in the AE < —0.15 GeV region that is mainly 
due to B Dtt, D — > Ktttt decays. This decay mode produces the same final state as the studied process plus one 
extra pion that is not included in the energy difference calculation. The decay B —f Dtt, D —f K^v^ also contributes 
due to muon-pion misidentification. The shape of this background is well described by MC simulation. The signal 
yields from fits are given in Table n 



B ~* KKK 



The dominant background from other B decays to the three-kaon final states is from B Dh decays, where h 
stands for a charged pion or kaon. To suppress this background, we reject events where the two-particle invariant 
mass is consistent within 15 MeV/c^ (--2.5cr) with D° —> K+K^ for the K+K+K^ final state and with D+ K°K+ 
for the K'^K'^K^ final state. To suppress the background due to tt/K misidentification in the K^K^K^ final state, 
we also exclude candidates if the invariant mass of any pair of oppositely charged tracks from the B candidate is 
consistent with — > K~tt'^ within 15 MeV/c^ (~ 2.5(t), independently of the particle identification information. 
For the K'^K^K^ final state we exclude candidates if the K^h^ invariant mass is consistent with £)+ K^tt^ 
within 15 MeV/c^. We also reject events with a K^K^ invariant mass that is consistent with Xco ^ K^K^ within 
50 MeV/c^. We do not find any charmless B decay modes that produce a significant background to the three-kaon 
final states. The feed-across between if+^^V+Tr^ and K'^^^^ final states is also found to be negligible. 

The AE distributions for all three-kaon final states are shown in Fig. where data (points with errors) are shown 
along with the expected background (hatched histograms). The BB background in KKK final states is much smaller 
than that in the +("^7r+7r^ final states and has no prominent structures. The results of fits to the AE distributions 
are summarized in Table While fitting the data for all three-kaon final states we fix not only the shape but also the 
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FIG. 3: A_B distributions for B —f KKK thrcc-body final states. Points with error bars are data; the open histogram is the fit 
result; the hatched histogram is the background. The straight line shows the qq continuum background contribution. 



normalization of the BB related background. The statistical significance of the B^ KgKgKg signal, in terms of the 
number of standard deviations is 4.3fT. It is calculated as 21n(£o/'Cmax), where £max and jCq denote the maximum 
likelihood with the nominal signal yield and with the signal yield fixed at zero, respectively. The significance of the 
signal in all other three-kaon final states exceeds 10a. 

B+(°) ^ if+(o)i^-7r+, K°sK°sn+, K+K+n- AND K-n+n+ 

The signals in the K~^^^^K~'k~^ and KgKgTr~^ channels are, in general, expected to be much smaller since the 
dominant contributions to these final states are expected to come from the b ^ u tree and b ^ dg penguin transitions, 
while the ^ K+'-'^^n+n- and B — !■ KKK decay channels are dominantly b ^ sg penguin transitions. As for 

the _B+ i^T+i^+TT" and B^ K~tt^tt~^ decays, the SM predicts exceedingly small branching fractions for these 
decays (as mentioned above). 

The background from other B meson decays to these final states is mainly due to B — > Dh decays, where h 
represents a charged pion or kaon. We suppress this background by rejecting events where the two-particle invariant 
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FIG. 4: Results of the fit (open histogram) to the Mbc distributions in AE bins. The data (points with error bars) arc 
compared with the BB MC expectation (hatched histogram). The feed-down from KKK and Kv-k final states is shown by 
filled histograms. 
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mass is consistent within 15 MeV/c^ (-2.5cr) with D° -> K+K', D° K-n+, D+ K°ti+ or D+ -> K^K+. 
Because of the small expected signal to background ratio for these final states, a different technique for signal yield 
extraction is used. We subdivide the AE region into 20 MeV bins and determine the signal yield in each bin from the 
fit to the corresponding Mbc spectrum. The A/bc signal shape is parameterized by a Gaussian function. The width of 
the Afbc signal is primarily due to the cm. energy spread and is expected to be the same for each decay channel; in the 
fit we fix it at the value cTMbc — 3.0 MeV/c^ determined from the — > C7r+, D'-^ K^ti^ events in the same data 
sample. The background shape is parameterized with the empirical function /(A/bc) a;VT— a?exp[— ^(1 — x^)], 
where x = Afbc/^^beam and ^ is a parameter fl^. We fix the ^ value from a study of data taken below the T(4iS') 
resonance. The signal yield from the A/bc fit as a function of A/? is used in the subsequent analysis. Since the qq 
background does not peak in the A/bc distribution, this technique allows an effective subtraction of the qq background. 
In contrast, the BB background can easily produce a signal-like distribution in the A/bc variable. The same procedure 
is applied to the BB MC events to determine the background shape and level. The resulting continuum-background- 
subtracted A/? distributions are fit with the signal yield as the only free parameter. Figure 01 shows the results of the 
fit, along with the expected contributions from generic BB decays and the feed-down due to particle misidentification 
from the B Ktttt and B KKK decay modes. For the K^-k^ti^ final state, shown in Fig. 0fa), the only 
background from charmless B decays is due to K^tt^ two-body decay. For the K^K^tt^ final state, shown 

in Fig. 0^c), we observe an excess of events in the signal region with statistical significance of 4a. For all other final 
states presented in Fig. 01 the experimental points are consistent with the background expectations. 



BRANCHING FRACTION CALCULATION 

To determine branching fractions, we normalize our results to the observed B~^ _D°7r+, /?" K~^tt~ and /?" 
D~Tr~^, D~ K^Ti~ signals. This reduces the systematic errors associated with the charged track reconstruction 
efficiency, particle identification efficiency, the event shape variables and uncertainty due to the possible nonuniform 
data taking conditions during the experiment. We calculate the branching fraction for B meson decay to a particular 
final state / via the relation 

B{B^ f)^^'-^-KB{B^D7:)B{D^K^), (1) 

where Nf and Ndt^ are the number of reconstructed signal events for the final state / and that for the Dn reference 
process, and £/ and sd-k are the corresponding reconstruction efficiencies determined from MC. The B Dn and 
D Kit branching fractions are the world average values from the PDG Q . 

To select B Dir events, we require that the Ktt invariant mass be consistent within 3 a with the D meson 
nominal mass 8]. The AE distributions for the reference processes /?+ — ^ /)°7r+, D'^ K^tt^ and B'^ D^tt^, 
K^TT^ are shown in Fig.O The results of the fits are given in Table Since, in the analysis of Ktttt (KKtt) 
and KKK final states, different requirements on the Fisher discriminant are applied, we determine the B Dtt 
signal yields for these two cases. 

The results of the three-body branching fraction measurements are presented in Table To determine the recon- 
struction efficiencies for K^K'^K^ and K'^K^K^ final states, we use a simple model _5] that takes into account the 
non-uniform distribution of signal events over the Dalitz plot. The three-body signal in this model is parameterized by 
a (pK intermediate state and a fxK state, where fx is a hypothetical wide scalar state. Another model is used for the 
^+(0)7^+7^- final states to account for the non-uniform distribution of signal events over the Dalitz plot and determine 
the reconstruction efficiencies 0. The three-body /f +*^"'7r+7r^ signal in this model is parameterized by the following 
set of quasi-two-body intermediate states: p°K, fo{980)K, K*{892)tt, K^{U30)tt and /o(1370)7r. For the K-tt+tt+, 
K'^ KgKg, KgKgKg and all KKtt final states, the reconstruction efficiency is determined from MC simulated events 
that are generated with a uniform (phase space) distribution over the Dalitz plot. For the /?"*" K~^K~tt~^ decay the 
statistical significance of the signal barely exceeds 4 a and as a final result we set a 90% confidence level upper limit 
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AE (GeV) AE (GeV) 

FIG. 5: AE distributions for B+ ^ I)°7r+, D° K+n' (left) and B° D--K+, D' K°n- (right) events. Points with 
error bars represent data with > 0, the open histogram is the fit result and the hatched histogram is the background. The 
straight line shows the qq continuum background contribution. 

TABLE I: Summary of results for B meson decays to three-body charmless hadronic final states. Table lists the three-body 
decay modes, the corresponding AE resolution for the core (iri) and the tail ((T2) Gaussian functions, the fraction of the signal 
in the core Gaussian, three-body reconstruction efficiency, the signal yield extracted from the AE fit and branching fractions. 
The branching fractions and 90% confidence level (CL) upper limits (UL) are quoted in units of 10~^. For the modes with 
neutral kaons the quoted reconstruction efficiencies include the intermediate branching fractions. Two values for the B Dn 
efficiency and signal yield correspond to > 0.8 (> 0) requirement on the Fisher discriminant. 



Three-body 


AE resolution ai/a2 


fraction 


Efficiency 


Signal Yield 


B or 90% CL UL 


mode 


(MeV) 




(%) 


(events) 


(io-«) 


K+Tl + TT- 


17.3/35.0 


0.85 


21.5 ±0.48 


845 ± 46 


53.6 ±3.1 ±5.1 




15.2/40.0 


0.85 


5.85 ±0.11 


209 ± 21 


45.4 ±5.2 ±5.9 


K+K+R- 


14.5/40.0 


0.85 


23.5 ± 0.50 


565 ± 30 


32.8 ±1.8 ±2.8 


K°K+K- 


14.0/40.0 


0.85 


7.20 ±0.17 


149 ± 15 


28.3 ±3.3 ±4.0 


K+KlKl 


14.3/40.0 


0.85 


6.78 ±0.19 


66.5 ± 9.3 


13.4 ± 1.9 ± 1.5 




14.7/40.0 


0.85 


3.98 ±0.17 


12.2l^j 


4.2+11 ± 0.8 


K+K-TT+ 


15.5/40.0 


0.85 


13.8 ±0.31 


94 ± 23 (< 130) 


9.3 ± 2.3 ± 1.1 (< 13) 




14.7/40.0 


0.85 


4.53 ±0.16 


27± 17 (< 55) 


< 18 




15.0/40.0 


0.85 


5.31 ±0.15 


-1.8 ±7.7 (< 11) 


< 3.2 


K-TV+TV + 


17.0/40.0 


0.85 


17.0 ±0.37 


21 ± 18 (< 51) 


< 4.5 


K+K+TT- 


15.5/40.0 


0.85 


14.2 ±0.30 


6.5 ±9.6 (< 22.3) 


< 2.4 




14.5/30.0 


0.80 


17.8 ± 0.43 (28.8 ± 0.57) 2470 ± 51 (4000 ± 66) 




D-TT + 


14.9/40.0 


0.80 


5.10 ±0.11 (8.24 ±0.14) 


300 ± 16 (451 ± 25) 





on its branching fraction, though the central value is also given in Table ^ Since we do not observe a statistically 
significant signal in any of the if~7r+7r+, K+K+Tr~ , K^K+tt^ and K^K^tt+ final states, we place 90% confidence 
level upper limits on their branching fractions |l7| . These limits are given in Table To take into account the 
systematic uncertainty, we reduce the reconstruction efficiency by one standard deviation of the overall systematic 
error. 

The dominant sources of systematic error are fisted in Table M For the S+(o) K+^^^ TT+TT and B+^^' 
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TABLE II: List of systematic errors (in percent) for the three-body branching fractions. 



Source 


1 

+ 
+ 


1 

+ 
o 


1 

+ 

+ 


1 

+ 

o 


+ 


OCT) 


+ 
+ 


1 

+ 

o 


-|- 

i< 


+ 
+ 
1 


+ 

+ 

< 


B Dn, D ^ Kn 


6.3 


11.7 


6.3 


11.7 


6.3 


11.7 


6.3 


11.7 


6.3 


6.3 


6.3 


Branching fractions 
























Eff. non-uniformity 


4.3 


3.5 


2.2 


3.6 


5.9 


7.2 


5.3 


5.7 


6.1 


6.8 


5.3 


over the Dalitz plot 
























Signal yield 


4.9 


3.6 


2.1 


5.6 


4.8 


7.6 


6.8 










extraction 
























PID 






4.0 


4.0 


2.0 


2.0 


2.0 


2.0 






2.0 


reconstruction 










3.5 


7.0 






3.5 






MC statistics 


3.3 


2.9 


2.9 


2.9 


3.4 


4.6 


3.3 


4.1 


3.7 


3.3 


3.2 


Total 


9.6 


13.1 


8.6 


14.3 


11.2 


17.9 


11.4 


13.8 


10.1 


9.8 


9.1 



K+^^^K+K- final states, we estimate tlie systematic uncertainty due to variations of reconstruction efficiency over 
the Dalitz plot by varying the relative phases of quasi-two-body states in the range from to 2tt and their amplitudes 
vifithin ±20%. For all other final states we make a conservative estimate of this type of systematic uncertainty. For 
each three-body final state the Dalitz plot is subdivided in 2 GeV^/c^ wide slices in M'^{hh) (for example M'^{K^tt^) 
for the K~^K~TT^ final state), and the reconstruction efficiency is determined in each bin. The same procedure is then 
applied for the second Dalitz plot projection {]VP{K^ K~) for the K^K~tt~^ decay). The maximal variation 

in the reconstruction efficiency for all bins is taken as the systematic error. The uncertainty due to the particle 
identification is estimated using high purity samples of kaons and pions from the Z?*^ —>■ K^tt^ decays, where the 
flavor is tagged using D*+ Z?°7r+ decays. The systematic error due to uncertainty in the Kg reconstruction 
efficiency is estimated from the comparison of the relative yields of inclusive Kg^s in off-resonance data and qq MC 
with variation of the A'^ selection criteria. We estimate the uncertainty due to the signal AE shape parameterization 
by allowing the width of the main Gaussian function to float and varying other parameters of the fitting function 
within their errors. The uncertainty in the background parameterization is estimated by varying the relative fractions 
of the background components and parameters of the qq background shape function within their errors. The overall 
systematic uncertainty for the three-body branching fractions varies from 9% to 18%, as given in Table UTI 



IMPLICATION FOR CP VIOLATION STUDY 



An important check of the Standard Model would be provided by measurements of the same CP-violating parameter 
in different weak interaction processes. A good example is the comparison of the measurement of the coefficient of 
the CP-violating sm{Amdt) term in the time dependent analysis of neutral B meson decays. In B'^ {cc)K'^ decays 
(where (cc) denotes a charmonium state) this coefficient is sin 20i . Precise measurements of sin 20i (also referred to 
as sin 2/3) have recently been reported by the Belle and BaBar experiments . The best known candidates for ^ s 
penguin-dominated processes, where this quantity can be measured independently, are B^ 4)K^ and B^ ■q'K'^ 
decays. However, the branching fractions for these decay modes are of order 10^^ — 10^^ (including secondary 
branching fractions) and very large numbers of B mesons are required to perform these measurements. This is 
especially true for the (pK'^ final state. The large signal observed in the three-body B^ KgK~^K~ decay mode, 
where the 4>Kg two-body intermediate state gives a relatively small contribution j^, Q| , would significantly increase 
the available statistics if these events could be used. Two complications are involved: (1) the b ^ u tree contributions 
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may introduce an additional weak phase in the KgK'^K~ amphtude and comphcate the interpretation of any 

observed CP violation. The b ^ u contribution in 5° — > 4>Kg is expected to be negligible (since is almost a pure 
ss state), which is not necessarily the case for the three-body K^K^K~ final state; (2) In contrast to the (pK'g state, 
which has fixed CP-parity, the CP-parity of the three-body KgK~^K^ final state is not a priori known. If the fractions 
of CP-even and CP-odd components are comparable, the use of the KgK^K^ decay mode in a CP analysis will be 
complicated. Although in this case an analysis of the proper time distribution would not be useful for CP violation 
measurement, the analysis of the time evolution of the Dalitz plot may still provide useful information on CP violation 
in the KgK~^K^ final state. In what follows we discuss the possibility of using the three-body i?" KgK^K^ 
decay mode for CP violation measurements jioi . 

The decays of B mesons to three-body Khh final states can be described by 6 — > u tree- level spectator and b s{d)g 
one-loop penguin diagrams. Although electroweak penguins, b ^ u W^-exchange, and annihilation diagrams can also 
contribute to these final states, they are expected to be much smaller and we neglect them in the following discussion. 
B meson decays to final states with odd numbers of kaons (s-quarks) are expected to proceed dominantly via the 
6 — > S(7 penguin transition since, for these states, the 6 — > m tree contribution has an additional CKM suppression. In 
contrast, B decays to three-body final states with two kaons proceed via the b —t u tree and b ^ dg penguin transitions 
with no 6 — *■ penguin contribution. This allows us to make a rough estimate of the 6 — > m tree contribution to 
final states with three kaons via the analysis of KKtt final states. This is illustrated for the — > K~^K~^K~ decay 
in Fig.0 where the dominant 6 — > s penguin graph is shown in Fig.|^a). The b ^ u tree graph (Fig. |Sfb)) has an 
additional Cabibbo suppression from the — > su vertex. The corresponding diagram without Cabibbo suppression 
(VF+ — !■ du) shown in Fig. Eld) is expected to be the dominant contributor to the K^K^t:^ final state. Assuming 
factorization, a quantitative estimate of the fraction of the b ^ u tree amplitude is then provided by the ratio 

where A^^^J^ is the total amplitude for the _B+ K^K^K^ decay and AjfJfJ^ is its 6 ^ u tree contribution. The 
(/i^/Zir)^ factor, where /tt = 131 MeV and Jk — 160 MeV are the pion and kaon decay constants, respectively, 
takes into account corrections for SU(3) breaking effects in the factorization approximation. 9c is the Cabibbo angle 
(sin 6*0 = 0.2205 ± 0.0018) jl]. Using the results for B+ ^ K+K-Tr+ and B+ K+K+K' branching fractions 
from Table n we obtain F « 0.022 ± 0.005. Similarly, for B° decays to K^K+R- and K'^K+tt- final states we find 
F K, 0.023 ± 0.013 (< 0.037), where the value in brackets is obtained using the upper limit for the B'^ K'^K^t:^ 
branching fraction. Thus, the contribution of the b ^ u tree transition is expected to be at the level of a few percent 
in branching fraction or at the level of 10 — 15% in amplitude. In contrast to two-body decays, where the magnitude 
of the interference term depends only on the relative phase between two amplitudes, in three-body decay, for the 
interference term to be maximal not only must the relative phase between the two amplitudes be (180) degrees at 
any point of the phase space, but these two amplitudes should also have identical behavior over the phase space. It 
seems unlikely that all these conditions will be satisfied since 6 — s- s penguin and 6 — > u tree amplitudes are quite 
different in nature. We conclude that the above value is a rather conservative estimate of the b u contribution. 

Let us now consider the CP content of the K^K^K^ state in neutral B decays. The CP-parity of the KgK^K^ 
three-body system is (—1)' , where I' is the orbital angular momentum of the K^K^ pair relative to the remaining 
neutral kaon. Since the total angular momentum of the K^K^K~ system is zero, V is equal to the relative orbital 
angular momentum, /, of the two charged kaons. Thus, the relative fraction of CP-even and CP-odd states in the 
K^K^K~ final state is determined by a fraction of states with even and odd orbital angular momenta in the K'^K~ 
system. This fraction could be determined by amplitude analysis of the B^ KgK'^K^ Dalitz plot. Such an analysis 
requires high statistics and cannot be performed with the available data. Instead, we use isospin relations between 
the different three-kaon final states to determine the relative fraction of CP-even and CP-odd states. Noting that the 
dominant b ~* sg penguin transition is an isospin conserving process, and neglecting the isospin violating b ^ u tree 
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FIG. 6: Diagrams for B'^ — > K'^K^K decay: (a) b s penguin; (b) and [c) b u trees, and for —> K^K vr"*" decay: 
(d) and (e) fe u trees; (i) b d penguin. 

and b dg penguin contributions, we can write the following relations: 

B{B" K°K+K-)^B{B+ ^ K+K"K")x (3) 

tb+ 

B{B° K^K°k°) = B{B+ K+K+K-) x (4) 

tb+ 

where the factor Tg+ /tb" = 1.091 ± 0.023 ± 0.014 22] takes into account the difference in total widths of charged and 
neutral B mesons. Being a mirror reflection of each other in isospin space, K^K^K^ and £?+ K^K'^K^ 

decays should have not only equal partial widths (Equation , but also the same decay dynamics; specifically, the 
fraction of a certain angular momentum state in the K~^K~ system in the K'^K'^K" final state should be the same as 
that for the K^K^ system in the K^K'^K^ final state. The i?+ — K^K^K^ decay produces three different observable 
states: K'^ K^K^, K^K'j^K^ and K'i.K^. The relative fractions of these states depend on the relative fractions 
of even and odd orbital angular momentum states in the K'^K^ system. Bose statistics requires that the K'^K^ wave 
function be symmetric (and, therefore, CP-even), independently of the relative orbital angular momentum, I, in the 
system of neutral kaons. As a result, a K'^K'^ system with Z-even can only decay to KgKg or K^K^ final states (with 
equal fractions), while a K'^K'^ system with l-odd can only decay to the KgKl final state (with an accuracy up to 
CP violation effects in the system of neutral kaons). 

In this analysis we reconstruct only the K^KgKg component of the K'^K'^K'^ final state. Measuring the 
K^K^K^ and B^ K^KgKg branching fractions and using the isospin relation (O we can determine 
the parameter a^, 

2 ^ ^B{B+_^K+iqK^ X ^ = N^+KOKO ^ . Skok+k- ^ tb^ ... 

B{BO^K"K+K-) TB+ N^OK^K- EK^KlKl Tb+' ^' 

Here, characterizes the fraction of states with even orbital angular momenta in the K^K^ system in the three-body 
K'^K'^K'^ final state. Due to isospin symmetry it also gives the fraction of states with even angular momenta in the 
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K+K- system for the KlK+K' final state. With the results given in Tabled we obtain: = 0.86 ± 0.15 ± 0.05, 
where we use the expression for in terms of signal yields [N) and reconstruction efficiencies (e) instead of branching 
fractions to reduce the systematic error due to uncertainty in branching fractions of the calibration modes. Note that 
the K'gK^K^ final state includes the (t>K^ state which is CP-odd. We remove B'^ events by requiring 

\M{K+K-) - > 15 MeV/c^; the number of remaining K^K+K^ events is 123 ± 14. The value for the 
remaining events is: a'^-^^^ ^ = 1.04 ±0.19 ±0.06. The fact that the a'^^n 4> value is close to unity provides evidence for 
the dominance of the CP-even component in the three-body KgK~^K~ final state when the 4>Kg intermediate state 
is excluded. 

From this analysis we conclude that the three-body — > KgK^K^ decay can be useful for the measurement 
of CP violation in 6 — > penguin dominated decays. Measurements of the time-dependent CP asymmetry in the 
b ^ sg penguin dominated _B" v'^s^ and i?° KgK^K^ decays are reported in Ref. 0|. 



CONCLUSION 



In conclusion, we have measured branching fractions for charmless B meson decays to the K^tt^tt" , K'^tt^tt" and 
three-kaon K^K^K^ , K^K^K^ , K^KgKg and KgKgKg final states. We also observe 4ct evidence for the signal 
in the K^K^tt^ final state that is expected to be dominated by the b u tree transition. We do not see any signal 
in B+ -> K-TT+TT+, B+ K+K+TT-, B° K°K+ti- and B+ KIK%-k+ decays and present 90% CL upper 
limit for their branching fractions. All the results on three-body branching fractions are summarized in Table The 
results presented in this work are in good agreement with our previous measurements 0| and with those reported by 
the BaBar and CLEO experiments. This paper does not update the results for quasi- two-body states obtained 
using the simplified technique in our previous analysis |^ because of the large model error of this technique. The 
extraction of branching fractions for exclusive quasi-two-body intermediate states in the observed three-body signals 
requires a full amplitude analysis of the corresponding Dalitz plots and is currently in progress. 

An isospin analysis of the charmless B meson decays to three-kaon final states suggests the dominance of the CP- 
even component in the three-body K^K^K^ final state when the (pK^ intermediate state is removed. Using this 
final state increases the statistics available for measurements of CP violation in & — > s penguin dominated decays by 
a factor of four compared to B^ (f'Kg. 
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